AI Governance & Responsible AI — Complete Quick Reference Guide

🏛 1. What Is AI Governance?

💡 Remember: AI Governance = Rules + Processes + People + Technology to manage AI responsibly

Definition

AI Governance is the system of rules, practices, processes, and technological tools that ensure an organization's use of AI technologies aligns with its strategies, objectives, legal obligations, and societal values.

It encompasses policies, oversight structures, accountability mechanisms, and technical guardrails across the entire AI lifecycle — from ideation to decommissioning.

Why It Matters (Business Case)

Regulatory Compliance — Avoid fines up to €35M or 7% global turnover (EU AI Act)
Trust & Reputation — 85% of consumers say trust influences AI product adoption
Risk Mitigation — AI incidents rose 56.4% from 2023→2024
Competitive Advantage — Governed AI = faster enterprise adoption
Liability Protection — Documented governance creates legal defensibility
Innovation Enablement — Clear boundaries accelerate safe experimentation

🧠 Mnemonic: "GREAT AI" — The 7 Dimensions of AI Governance

Governance Structure

Risk Management

Ethics & Fairness

Accountability

Transparency

Auditability

Inclusion & Privacy

€35M

Max EU AI Act fine

56%

YoY AI incident increase

127+

Countries with AI policies

Aug 2026

EU AI Act full enforcement

⚖️ 2. The Six Pillars of Responsible AI

💡 Remember: "FRIST-A" → Fairness, Reliability, Inclusiveness, Security, Transparency, Accountability

⚖️

1. Fairness

AI systems should treat all people equitably, avoiding discrimination based on race, gender, age, disability, or other protected characteristics. Includes equal quality of service and equitable impact assessment.

Bias Testing Demographic Parity Equal Opportunity

🛡️

2. Reliability & Safety

AI systems must perform reliably and safely under expected and unexpected conditions. Includes robustness testing, fail-safes, graceful degradation, and continuous monitoring.

Robustness Fail-safes Stress Testing

🤝

3. Inclusiveness

AI systems should be designed to engage and empower everyone, addressing potential barriers and ensuring accessibility across diverse populations, abilities, and contexts.

Accessibility Diverse Teams Universal Design

🔐

4. Privacy & Security

AI systems must protect user data, ensure confidentiality, prevent unauthorized access, and resist adversarial attacks. Data minimization and purpose limitation are core.

Data Minimization Encryption Access Control

👁️

5. Transparency

People should understand how AI systems work, what data they use, and how decisions are made. Includes documentation, model cards, and clear communication to all stakeholders.

Model Cards Data Sheets Explainability

📋

6. Accountability

Organizations and individuals must be answerable for AI systems they design, develop, and deploy. Requires clear ownership, audit trails, redress mechanisms, and governance committees.

Audit Trails Governance Board Redress

🏢 Industry Alignment: Microsoft uses all 6 pillars. Google emphasizes "Social Benefit" + "Avoiding Harm." IBM focuses on "Trust, Fairness, Robustness, Transparency, Privacy." Despite naming differences, the core principles converge across all major tech companies and international bodies (OECD, UNESCO, G7).

📐 3. Key Governance Frameworks Compared

💡 Remember: "NIE" — NIST (voluntary US), ISO (certifiable intl), EU AI Act (mandatory law)

Dimension	🇺🇸 NIST AI RMF 1.0	🌐 ISO/IEC 42001	🇪🇺 EU AI Act
Type	Voluntary framework / guidelines	Certifiable international standard	Binding legislation (regulation)
Scope	All AI systems; any organization	AI management system (AIMS) for any org	AI systems placed on or used in EU market
Structure	4 Functions: Govern → Map → Measure → Manage	PDCA cycle: Plan → Do → Check → Act (ISO management system)	Risk-based tiers: Prohibited → High → Limited → Minimal
Risk Approach	Context-dependent risk profiles	Formal risk assessment aligned to ISO 31000	Pre-defined risk categories with obligations
Enforcement	None (voluntary), but referenced by regulators	Third-party certification audits	Fines: up to €35M or 7% global turnover
Best For	Building internal AI risk culture & processes	Demonstrating maturity to clients & regulators	Legal compliance for EU market
Relationship	NIST maps to ISO 42001 — official crosswalk exists		ISO 42001 + NIST help satisfy EU AI Act conformity

NIST AI RMF — The Four Core Functions

🏛 GOVERN
Culture, policies, roles

→

🗺 MAP
Context & risk identification

→

📏 MEASURE
Assess & quantify risks

→

⚙️ MANAGE
Treat, monitor, improve

🏛 GOVERN (Cross-cutting)

Establish AI governance structure & committee
Define risk tolerances & organizational values
Assign roles: AI Risk Owner, Ethics Lead, Data Steward
Create policies for the full AI lifecycle
Foster a culture of responsible AI development
Ensure third-party/vendor AI governance alignment

🗺 MAP (Contextualize)

Identify AI system purpose, context, and stakeholders
Catalog all AI systems (AI inventory/registry)
Determine potential impacts (positive & negative)
Assess interdependencies with other systems
Identify legal/regulatory requirements per system
Document data lineage and model provenance

📏 MEASURE (Quantify)

Define metrics for trustworthiness characteristics
Test for bias, fairness, accuracy, robustness
Conduct adversarial testing & red-teaming
Perform privacy attack simulations
Evaluate explainability of outputs
Benchmark against established thresholds

⚙️ MANAGE (Act)

Prioritize and treat identified risks
Implement controls & guardrails
Deploy continuous monitoring & alerting
Establish incident response procedures
Track risk metrics over time (drift, decay)
Feed learnings back into Govern → Map cycle

🇪🇺 4. EU AI Act — Deep Dive

💡 Remember the risk pyramid: "BAN → HIGH → LIMITED → MINIMAL" — strictness flows top-down

🚫 UNACCEPTABLE RISK — BANNED
Social scoring, subliminal manipulation, real-time biometric in public, emotion recognition at work/school

⚠️ HIGH RISK — Strict Obligations
Critical infrastructure, education, employment, law enforcement, migration, justice, credit scoring, medical devices

ℹ️ LIMITED RISK — Transparency Required
Chatbots, deepfakes, emotion detection, AI-generated content must be labeled

✅ MINIMAL RISK — No Obligations
Spam filters, recommendation engines, video games, inventory management

🚫 Prohibited AI Practices (Banned since Feb 2025)

Cognitive manipulation exploiting vulnerabilities (age, disability)
Social scoring by governments (Chinese-style citizen rating)
Real-time remote biometric identification in public spaces by law enforcement (with narrow exceptions)
Biometric categorization inferring sensitive attributes (race, sexuality, political views)

Predictive policing based solely on profiling or personality traits
Untargeted facial image scraping from internet/CCTV for facial recognition databases
Emotion recognition in workplaces and educational institutions
Exploitation of vulnerabilities of specific groups to materially distort behavior

⚠️ High-Risk AI System Obligations

Obligation	What It Requires	Who Is Responsible
Risk Management System	Continuous identification, analysis, estimation & treatment of risks throughout lifecycle	Provider
Data Governance	Training data must be relevant, representative, free of errors; documented data lineage	Provider
Technical Documentation	Detailed docs on system design, development, capabilities, limitations before market placement	Provider
Record-Keeping	Automatic logging of events for traceability (minimum retention periods apply)	Provider + Deployer
Transparency	Clear instructions for deployers; users informed they are interacting with AI	Provider + Deployer
Human Oversight	Human-in-the-loop or human-on-the-loop controls; ability to override/shut down	Deployer
Accuracy & Robustness	Appropriate levels of accuracy; resilience to errors, faults, adversarial attacks	Provider
Cybersecurity	Protection against unauthorized access, data poisoning, model manipulation	Provider
Conformity Assessment	Self-assessment or third-party audit before market placement	Provider
Post-Market Monitoring	Ongoing surveillance; serious incident reporting to authorities	Provider

🤖 General-Purpose AI (GPAI) Model Rules

All GPAI Models

Maintain up-to-date technical documentation
Provide information & documentation to downstream providers
Comply with EU copyright law (including training data transparency)
Publish detailed summary of training data content

GPAI with Systemic Risk (>10²⁵ FLOPs)

All above + model evaluation including adversarial testing
Assess and mitigate systemic risks
Track & report serious incidents to EU AI Office
Ensure adequate cybersecurity protections
Report energy consumption of the model

📅 EU AI Act Enforcement Timeline

Aug 1, 2024

AI Act enters into force (20 days after publication)

Feb 2, 2025 ✅ PASSED

Prohibited AI practices banned + AI Literacy obligations apply

Aug 2, 2025 ✅ PASSED

GPAI transparency rules + governance structure operational

Aug 2, 2026 ⏳ UPCOMING

FULL ENFORCEMENT: High-risk AI systems must comply; full penalty regime active

Aug 2, 2027

Legacy high-risk AI in already-regulated products must comply

⚠️ 5. AI Risk Management

💡 Remember: "IMPACT" — Identify, Measure, Prioritize, Act, Control, Track

AI Risk Taxonomy

Risk Category	Examples	Impact	Mitigation Strategy
Technical	Model drift, data quality degradation, adversarial attacks, hallucinations	Wrong decisions, safety failures	Continuous monitoring, retraining triggers, adversarial testing
Ethical	Bias, discrimination, unfair outcomes, manipulation	Legal liability, reputational damage, societal harm	Fairness audits, diverse datasets, ethics review boards
Legal & Regulatory	Non-compliance with EU AI Act, GDPR violations, IP infringement	Fines, lawsuits, market bans	Compliance mapping, legal review, regulatory monitoring
Operational	Over-reliance on AI, loss of human expertise, single point of failure	Business disruption, skill atrophy	Human-in-the-loop, fallback procedures, skills training
Reputational	Public controversy, biased outputs reported by media, user backlash	Brand damage, customer churn	Crisis communication plans, proactive transparency
Security	Data poisoning, model theft, prompt injection, data exfiltration	Breach, data loss, compromised integrity	Security testing, access controls, input validation
Societal	Job displacement, misinformation at scale, concentration of power	Social instability, inequality	Impact assessments, stakeholder engagement, policy advocacy

🧠 AI Risk Assessment Matrix (Severity × Likelihood)

Low Likelihood

Medium

High

Very High

Critical Severity

🟠 High

🔴 Critical

High Severity

🟡 Medium

🟠 High

🔴 Critical

Medium Severity

🟢 Low

🟡 Medium

🟠 High

🔴 Critical

Low Severity

🟢 Low

🟡 Medium

🟠 High

🎯 6. AI Bias & Fairness — The Complete Guide

💡 Remember: Bias can enter at EVERY stage — Data → Algorithm → Deployment → Feedback Loop

Types of AI Bias

Bias Type	Description	Real-World Example	Stage
Selection Bias	Training data not representative of real-world population	Medical AI trained mostly on Caucasian patient data misdiagnoses for darker skin	Data
Historical Bias	Data reflects past societal prejudices	Amazon recruiting tool penalized resumes with "women's" — trained on 10 yrs of male-dominated hiring	Data
Measurement Bias	Proxy variables correlate with protected attributes	Using zip code as proxy for creditworthiness discriminates by race	Data
Label Bias	Inconsistent or subjective labeling of training data	Annotators labeling African-American Vernacular English as "toxic" more often	Data
Aggregation Bias	Single model for diverse subgroups	Diabetes prediction models optimized for majority population fail for minorities	Algorithm
Algorithmic Bias	Model architecture or optimization amplifies disparities	COMPAS recidivism tool: higher false positive rate for Black defendants	Algorithm
Confirmation Bias	System reinforces existing patterns	Content recommendation creating filter bubbles and radicalization pathways	Deployment
Automation Bias	Humans over-trust AI decisions	Clinicians accepting AI diagnosis without independent verification	Deployment
Feedback Loop Bias	Biased outputs reinforce biased inputs over time	Predictive policing: more patrols → more arrests → "validates" model → more patrols	Feedback
Representation Bias	Underrepresentation in dev teams leads to blind spots	Voice assistants performing poorly for non-native English speakers	Process

Fairness Metrics — Know These!

Group Fairness Metrics

Demographic Parity: P(Ŷ=1|A=0) = P(Ŷ=1|A=1) — Equal positive prediction rates across groups
Equalized Odds: Equal TPR and FPR across groups — same accuracy for each group
Equal Opportunity: Equal TPR across groups — qualified people from all groups equally likely to be selected
Predictive Parity: Equal PPV across groups — positive predictions equally accurate
Calibration: Predicted probabilities match actual outcomes within each group

⚠️ Impossibility Theorem: You generally CANNOT satisfy all fairness metrics simultaneously (Chouldechova, 2017). Choose based on your use case context!

Bias Mitigation Techniques

Pre-processing: Reweighting, resampling, relabeling training data; removing or transforming sensitive features
In-processing: Fairness constraints during training (adversarial debiasing, fair regularization); constrained optimization
Post-processing: Threshold adjustment per group; calibrated equalized odds; reject-option classification
Organizational: Diverse development teams, external audits, community engagement, bias bounty programs

🛠 Key Tools: IBM AI Fairness 360, Google What-If Tool, Microsoft Fairlearn, Aequitas, Amazon SageMaker Clarify

🔍 7. Explainability & Interpretability (XAI)

💡 Remember: "SLICE" — SHAP, LIME, Intrinsic models, Counterfactuals, Execution graphs

Technique	Type	How It Works	Scope	Best For
SHAP (SHapley Additive exPlanations)	Model-agnostic	Uses game theory (Shapley values) to assign contribution of each feature to prediction	Local + Global	Feature importance, regulatory explanations, any ML model
LIME (Local Interpretable Model-agnostic Explanations)	Model-agnostic	Creates local linear approximation around a specific prediction by perturbing inputs	Local only	Explaining individual predictions to end-users
Attention Maps	Model-specific	Visualizes which parts of input the model "attends to" (transformers, CNNs)	Local	NLP, image classification, debugging
Counterfactual Explanations	Model-agnostic	"What would need to change for a different outcome?" — smallest change to flip the decision	Local	Loan denials, job rejections — actionable feedback
Partial Dependence Plots	Model-agnostic	Shows marginal effect of one or two features on predicted outcome	Global	Understanding feature relationships
Decision Trees / Rule Lists	Intrinsically interpretable	Model itself is human-readable — series of if-then rules	Global	High-stakes domains requiring full transparency
Model Cards	Documentation	Standardized disclosure of model purpose, performance, limitations, ethical considerations	Global	Transparency for all stakeholders
Confidence Scores	Output-level	Probability or certainty level attached to each prediction	Local	Enabling human override when confidence is low

📋 When to Use What: Use SHAP for regulatory/audit explanations (mathematically grounded). Use LIME for quick end-user explanations. Use Counterfactuals when users need actionable feedback. Use Intrinsic models (decision trees, linear models) for highest-stakes decisions where full transparency is non-negotiable.

SHAP vs LIME — Feature Comparison

SHAP Strengths

Mathematical rigor

95%

Global explanations

90%

Consistency

95%

Computation speed

40%

LIME Strengths

Simplicity

90%

Speed

85%

User-friendly

88%

Stability

50%

🔒 8. AI Security Threats & Defenses

💡 Remember: "PADME" — Poisoning, Adversarial inputs, Data exfiltration, Model theft, Evasion

🚨 OWASP 2025 Top 10 for LLM Applications: #1 Prompt Injection, #2 Sensitive Information Disclosure, #3 Supply Chain Vulnerabilities, #4 Data and Model Poisoning, #5 Improper Output Handling, #6 Excessive Agency, #7 System Prompt Leakage, #8 Vector and Embedding Weaknesses, #9 Misinformation, #10 Unbounded Consumption

Threat	Description	Attack Vector	Defense
Prompt Injection #1 OWASP	Attacker crafts inputs to override system instructions or extract sensitive info	Direct (user input) or indirect (poisoned content the model reads)	Input sanitization, instruction hierarchies, output filtering, canary tokens, system prompt hardening
Data Poisoning	Corrupting training data to introduce backdoors or degrade model performance	Compromised training pipelines, web scraping of adversarial content	Data provenance tracking, anomaly detection on training data, robust training methods
Adversarial Examples	Subtle perturbations to inputs that cause misclassification	Pixel-level image changes, word substitutions in text	Adversarial training, input preprocessing, ensemble methods, certified defenses
Model Inversion	Extracting training data or sensitive info from model outputs	Querying model repeatedly to reconstruct private training examples	Differential privacy, output perturbation, rate limiting, membership inference detection
Model Theft / Extraction	Replicating a model's behavior through API queries	Systematic querying to build a surrogate model	Rate limiting, watermarking, query monitoring, response perturbation
Supply Chain Attack	Compromised pre-trained models, libraries, or datasets	Trojanized models on Hugging Face, malicious pip/npm packages	Model scanning, SBOM (Software Bill of Materials), verified model sources
Data Exfiltration	LLM reveals PII, credentials, or proprietary info from training data or context	Crafted prompts, jailbreaks, context window exploitation	PII detection/masking, output scanning, DLP integration, context isolation
Excessive Agency	AI system takes unauthorized actions beyond intended scope	Tool-calling without proper constraints, autonomous code execution	Least privilege, action approval gates, sandboxing, human-in-the-loop

Defense-in-Depth Architecture for AI Systems

🧱 Input
Validation

→

🔐 Auth &
Access Control

→

🛡 Model
Guardrails

→

📊 Output
Filtering

→

📝 Logging &
Monitoring

→

🚨 Incident
Response

🛡 9. AI Privacy & Data Protection

💡 Remember: "CLIP-D" — Consent, Lawful basis, Information minimization, Purpose limitation, Data subject rights

GDPR & AI — Key Intersections

Art. 22 — Automated Decision-Making: Right not to be subject to solely automated decisions with legal/significant effects; right to human intervention & explanation
Art. 13-14 — Transparency: Must inform data subjects about AI processing logic, significance, and consequences
Art. 5(1)(c) — Data Minimization: Only collect data necessary for the AI system's purpose
Art. 5(1)(b) — Purpose Limitation: Data collected for one purpose cannot be freely repurposed for AI training
Art. 35 — DPIA Required: Data Protection Impact Assessment mandatory for high-risk AI processing
Art. 17 — Right to Erasure: "Machine unlearning" — removing data from trained models is technically challenging

Privacy-Preserving AI Techniques

Differential Privacy: Adding calibrated noise to data or outputs to prevent identification of individuals (used by Apple, Google Census)
Federated Learning: Training models across decentralized devices without sharing raw data (used by Google Keyboard, Apple Siri)
Homomorphic Encryption: Computing on encrypted data without decrypting it — ultimate privacy but very slow
Secure Multi-Party Computation (SMPC): Multiple parties jointly compute a function without revealing their inputs
Synthetic Data Generation: Creating artificial data that preserves statistical properties without real PII
Data Anonymization & Pseudonymization: K-anonymity, L-diversity, T-closeness techniques
Trusted Execution Environments (TEE): Hardware-isolated secure enclaves for sensitive AI computation

⚠️ Key Precedent: In early 2025, OpenAI was fined €15M by Italy's DPA for training models on personal data without clear legal basis and lacking age verification. This signals aggressive enforcement of GDPR against AI companies.

🔄 10. AI Governance Across the Lifecycle

💡 Remember: "DDTDM-D" — Design, Data, Train, Deploy, Monitor, Decommission

Phase	Governance Activities	Key Artifacts	Responsible Roles
1. Problem Formulation & Design	Purpose assessment, stakeholder impact analysis, risk classification (EU AI Act tier), ethics review, necessity & proportionality check, defining fairness criteria	AI Use Case Assessment, Ethics Review Form, Risk Classification Report	Product Manager, Ethics Board, Legal
2. Data Collection & Preparation	Data lineage documentation, consent verification, bias auditing of datasets, DPIA (if high-risk), data quality assessment, representativeness check	Data Card / Datasheet, DPIA Report, Consent Records, Data Lineage Map	Data Engineer, DPO, Data Scientist
3. Model Development & Training	Fairness-aware training, adversarial testing, privacy-preserving techniques, explainability method selection, experiment tracking, IP/copyright compliance	Model Card, Training Report, Fairness Assessment, Experiment Logs	ML Engineer, Data Scientist, Security Engineer
4. Testing & Validation	Red-teaming, bias testing across subgroups, robustness evaluation, security penetration testing, performance benchmarking, human evaluation	Test Reports, Red Team Findings, Conformity Assessment (EU AI Act)	QA, Security Team, Ethics Reviewer
5. Deployment	Human oversight mechanisms, user disclosure (AI transparency), monitoring setup, rollback procedures, incident response readiness, access controls	Deployment Checklist, Monitoring Dashboard, Incident Response Plan, User Disclosure	DevOps/MLOps, Product Manager, Legal
6. Monitoring & Maintenance	Performance drift detection, fairness metric tracking, adversarial monitoring, user feedback analysis, incident logging, periodic re-assessment	Monitoring Reports, Drift Alerts, Incident Logs, Periodic Review Reports	MLOps, Data Scientist, AI Risk Owner
7. Decommissioning	Sunsetting plan, data deletion/archival (GDPR compliance), user notification, documentation archival, model artifact disposal, knowledge transfer	Decommissioning Report, Data Disposal Certificate, Archive Records	Product Manager, DPO, Engineering

👥 11. Governance Roles & Responsibilities (RACI)

💡 A strong governance org has: Board oversight → Ethics committee → Cross-functional working groups

Role	Key Responsibilities	Governance Focus
Chief AI Officer (CAIO)	Overall AI strategy, governance oversight, board reporting	Strategic direction, risk appetite, resource allocation
AI Ethics Board	Review high-risk use cases, set ethical guidelines, handle escalations	Ethics policies, use case approvals, societal impact
Product Manager	Define AI product requirements, manage stakeholder expectations, ensure responsible use cases	Use case justification, user impact, fairness requirements, labeling/transparency
Data Protection Officer (DPO)	GDPR/privacy compliance, DPIAs, data subject rights	Privacy by design, consent, data minimization, erasure
AI Risk Manager	Identify, assess, and track AI risks; maintain risk register	Risk taxonomy, mitigation tracking, incident management
Security Engineer	Secure AI pipelines, adversarial defense, access controls	Threat modeling, penetration testing, supply chain security
ML Engineer / Data Scientist	Model development, fairness testing, explainability implementation	Bias mitigation, model cards, experiment reproducibility
Legal / Compliance	Regulatory mapping, contractual obligations, liability assessment	EU AI Act classification, GDPR compliance, IP rights
Internal Audit	Independent verification of governance controls	Audit trails, conformity assessments, effectiveness reviews

🧠 PM's AI Governance Checklist (What YOU Own)

☐ Is this use case ethically justified?

☐ Have we classified the risk tier (EU AI Act)?

☐ Who are the affected stakeholders?

☐ What fairness criteria apply?

☐ Is there human oversight built in?

☐ Are users informed they're interacting with AI?

☐ What happens when the model is wrong?

☐ Is there a redress mechanism?

☐ Have we done an impact assessment?

☐ Is the model card / documentation complete?

🌍 12. Global AI Regulatory Landscape

💡 Remember: EU = Regulation-first, US = Sector-specific, China = State-controlled, UK = Pro-innovation

Region / Country	Key Framework	Approach	Status (2026)
🇪🇺 European Union	EU AI Act + GDPR + AI Liability Directive	Risk-based, comprehensive regulation with strict enforcement	Enforcing Full high-risk rules from Aug 2026
🇺🇸 United States	NIST AI RMF + Executive Orders + State laws (CO, IL, CA)	Voluntary frameworks + sector-specific rules + state-level legislation	Evolving Patchwork of federal + state regulations
🇬🇧 United Kingdom	Pro-Innovation AI Framework + AI Safety Institute	Principles-based, sector regulators apply existing rules to AI	Active AI Safety Institute operational
🇨🇳 China	Generative AI Measures + Algorithm Recommendation Rules + Deep Synthesis Rules	Specific regulations per AI type; state content control; mandatory registration	Enforcing Multiple specific laws active
🇨🇦 Canada	AIDA (Artificial Intelligence and Data Act) — Bill C-27	Risk-based classification similar to EU; criminal penalties for reckless AI	Pending Still in legislative process
🇮🇳 India	Digital India Act (proposed) + DPDP Act 2023	Balancing innovation with data protection; no dedicated AI law yet	Developing Advisory-based approach
🇧🇷 Brazil	AI Bill (PL 2338/2023)	Risk-based framework inspired by EU AI Act	Advancing Expected passage soon
🇯🇵 Japan	AI Guidelines for Business + Hiroshima AI Process	Soft law / voluntary guidelines; industry self-regulation	Active Voluntary guidelines in effect
🌐 International Bodies	OECD AI Principles, UNESCO Recommendation, G7 Hiroshima Process, ISO/IEC 42001	Setting global norms, principles, and standards for harmonization	Active Influencing national laws globally

🤖 13. Generative AI — Special Governance Considerations

💡 GenAI multiplies governance challenges: hallucinations, copyright, deepfakes, prompt attacks

Unique GenAI Risks

🎭 Hallucinations & Confabulation

GenAI models confidently generate false, fabricated, or nonsensical information. This is especially dangerous in healthcare, legal, and financial domains.

Implement retrieval-augmented generation (RAG) for grounding
Add confidence scoring and uncertainty quantification
Require human review for high-stakes outputs
Provide source citations and traceability

Models trained on copyrighted content raise legal questions about training data use and generated output ownership.

Track and document training data sources
Implement content provenance (C2PA standard)
Monitor for verbatim reproduction of copyrighted text
EU AI Act requires training data summary publication
NYT v. OpenAI (2023) — landmark lawsuit still evolving

🎬 Deepfakes & Synthetic Media

AI-generated video, audio, and images that are indistinguishable from real content.

EU AI Act mandates labeling AI-generated content
Implement digital watermarking (SynthID, C2PA)
Deploy detection tools for synthetic media
Establish clear usage policies and terms of service

💉 Prompt Injection & Jailbreaking

Users craft inputs to bypass safety controls, extract system prompts, or make the model produce harmful content.

Multi-layered prompt defenses (system > user instruction hierarchy)
Input and output content filters
Red-team testing before deployment
Continuous monitoring for novel attack patterns
Rate limiting and abuse detection

📊 Environmental Impact

Training large models consumes enormous energy and water resources.

EU AI Act requires energy consumption reporting for GPAI with systemic risk
Track carbon footprint of training and inference
Consider model efficiency (distillation, quantization, smaller models)
Use renewable energy-powered data centers where possible

GenAI Governance Framework

Acceptable Use Policy (Must-Have)

Define approved use cases and prohibited uses
Set data input restrictions (no PII, confidential data without controls)
Require human review thresholds by risk level
Mandate output verification for external-facing content
Establish vendor assessment criteria for third-party AI
Define data retention and deletion policies

AI Content Labeling Requirements

EU AI Act (Art. 50): Users must be informed when interacting with AI; deepfakes must be labeled; AI-generated text for public info must be labeled
C2PA Standard: Content Credentials for provenance tracking — "nutrition label" for digital content
Watermarking: Google SynthID, Meta's invisible watermark for AI-generated images
Internal tagging: All AI-generated artifacts should carry metadata indicating AI involvement

🛠 14. AI Governance Tools & Platforms

💡 No single tool does everything — build a governance stack aligned to your framework

Category	Tool / Platform	Provider	Key Capabilities
Fairness & Bias	AI Fairness 360	IBM (Open Source)	70+ fairness metrics, 11 bias mitigation algorithms
	Fairlearn	Microsoft (Open Source)	Fairness assessment, bias mitigation for classification & regression
	What-If Tool	Google (Open Source)	Visual interface for exploring model fairness without code
Explainability	SHAP Library	Open Source	Shapley-based explanations for any ML model
	LIME Library	Open Source	Local interpretable explanations via surrogate models
	AI Explainability 360	IBM (Open Source)	8 explainability algorithms + tutorials
Governance Platforms	ModelOp Center	ModelOp	Enterprise AI governance, model inventory, risk scoring
	Credo AI	Credo AI	AI governance platform with policy packs for EU AI Act, NIST
	Holistic AI	Holistic AI	Auditing, risk management, compliance automation
Security	Garak	NVIDIA (Open Source)	LLM vulnerability scanner — prompt injection, data leakage
Security	Rebuff	Open Source	Prompt injection detection framework
Monitoring	Evidently AI	Open Source	ML model monitoring — drift, performance, data quality
Monitoring	Fiddler AI	Fiddler	Model performance management + explainability
Privacy	PySyft	OpenMined (Open Source)	Federated learning, differential privacy, encrypted computation
Privacy	Opacus	Meta (Open Source)	Differential privacy for PyTorch model training
Documentation	Model Card Toolkit	Google (Open Source)	Standardized model documentation generation

✅ 15. AI Governance Master Checklist

💡 Use this as a go/no-go gate before deploying any AI system

🏛 Governance & Organization

AI governance committee / ethics board established
AI policy and acceptable use policy documented
Roles & responsibilities clearly assigned (RACI)
AI system registry / inventory maintained
Third-party AI vendor assessment process in place
AI literacy training program for all employees
Escalation and exception handling procedures defined

⚖️ Ethics & Fairness

Use case ethics review completed
Stakeholder impact assessment performed
Fairness metrics defined and measured
Bias testing across protected groups completed
Diverse development team involved
Feedback and redress mechanism available to users
Regular fairness re-assessment scheduled

📋 Compliance & Legal

AI system risk classification completed (EU AI Act tier)
DPIA conducted (if required by GDPR)
Conformity assessment done (if high-risk)
Technical documentation prepared
User transparency/disclosure implemented
Copyright & IP review of training data completed
Post-market monitoring plan established
Incident reporting procedures in place

🔒 Security & Privacy

Threat model created for AI system
Adversarial testing / red-teaming completed
Prompt injection defenses implemented (for LLMs)
Data encryption at rest and in transit
Access controls and audit logging enabled
Privacy-preserving techniques applied where needed
Supply chain security (model provenance verified)
Incident response plan specific to AI scenarios

🔍 Transparency & Explainability

Model Card published with performance + limitations
Data Card / Datasheet documenting training data
Explainability method selected and implemented
Human oversight mechanism (HITL/HOTL) operational
Override/shutdown capability tested
AI-generated content labeled where required
Confidence scores available for predictions

📊 Monitoring & Operations

Model performance monitoring dashboards live
Data drift and concept drift detection enabled
Fairness metrics tracked over time
Retraining triggers defined and automated
User feedback collection and analysis loop
Regular governance review cadence established
Decommissioning plan documented

📖 16. Glossary of Key Terms

Term	Definition
AI System	A machine-based system that generates outputs (predictions, decisions, recommendations, content) that can influence environments. The EU AI Act defines it as a system designed to operate with varying levels of autonomy.
Algorithmic Impact Assessment (AIA)	A systematic evaluation of the potential impacts of an AI system on individuals, groups, and society before deployment.
Adversarial Attack	Intentionally crafted inputs designed to cause an AI model to make errors or behave unexpectedly.
Bias (Algorithmic)	Systematic and repeatable errors in AI output that create unfair outcomes for particular groups.
Conformity Assessment	The process of verifying that an AI system complies with regulatory requirements (EU AI Act requires this for high-risk systems).
Data Lineage	The complete tracking of data from its origin through all transformations to its use in model training and inference.
Deepfake	AI-generated synthetic media (video, audio, images) designed to appear authentic. Must be labeled under EU AI Act.
Differential Privacy	A mathematical framework that provides provable guarantees that individual data points cannot be identified in aggregate datasets.
DPIA	Data Protection Impact Assessment — required under GDPR Art. 35 for processing likely to result in high risk to individuals.
Explainability (XAI)	The degree to which the internal mechanics and outputs of an AI system can be understood by humans.
Federated Learning	A machine learning approach where models are trained across decentralized data sources without exchanging raw data.
GPAI	General-Purpose AI — AI models capable of performing a wide range of tasks (e.g., GPT-4, Claude, Gemini). Subject to specific EU AI Act obligations.
Hallucination	When a generative AI model produces content that is factually incorrect, fabricated, or inconsistent with training data, presented with false confidence.
Human-in-the-Loop (HITL)	A system design where a human must approve or intervene in AI decisions before they take effect.
Human-on-the-Loop (HOTL)	A system design where a human monitors AI decisions and can intervene or override, but AI acts autonomously by default.
Model Card	A standardized document describing a model's intended use, performance metrics, limitations, and ethical considerations.
Model Drift	The degradation of model performance over time as real-world data distributions change from training data.
Prompt Injection	An attack where malicious instructions are embedded in inputs to override an LLM's intended behavior.
RAG	Retrieval-Augmented Generation — technique that grounds LLM responses in retrieved factual documents to reduce hallucinations.
Red-Teaming	Adversarial testing where a team actively tries to find failures, biases, and security vulnerabilities in an AI system.
Responsible AI	The practice of designing, developing, deploying, and operating AI systems in a manner consistent with ethical principles, legal requirements, and societal values.
Shadow AI	Unauthorized or ungoverned use of AI tools by employees outside of IT/governance oversight — a major organizational risk.
Systemic Risk (GPAI)	Under EU AI Act, a GPAI model has systemic risk if trained with >10²⁵ FLOPs or designated by EU AI Office, triggering additional obligations.